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DIGITAL CAMERA AND LENS SYSTEM AND METHOD 
BACKGROUND OF THE INVENTION 

1 . FIELD OF THE INVENTION 

The system and method of the present invention relates to a computer 
implemented digitally simulated camera and lens system. 

2. ART BACKGROUND 

The use of computers to display graphic images has become widespread. 
Computers frequently are used to generate and /or display three-dimensional 
images, pictures and moving images such as video and movies. Animation 
using the computer is quite common. These images are generated a variety of 
ways. For instance, an image may be computer generated through software 
executing on the computer. At other times real world images are imported 
from another media, such as film or a camera and lens apparatus electrically 
connected to the computer system. Computers also are being used to combine 
real world images and computer generated images. 

One example is the use of computer generated images to provide special 
effects in video or motion pictures. Many films utilize computer generated 
characters that appear to walk around, talk and perform actions and appear to 
be part of a photographed real world wherein the environment was 
photographed separately and prior to the placement of the computer generated 
effects. In this type of situation, a high degree of accuracy is required to 
replicate the real world environment and replicate the lens characteristics for 
the incorporation of the computer generated effects. Problems can arise when 
combining real world images and computer generated images. 
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In order for the computer generated special effects images to match the 
real world photography, it is necessary to duplicate the lens characteristics and 
the motions of the camera which took the real world photographic images as 
closely as possible. If the camera lens setting is not duplicated accurately, the 
5 elements of the perspective will be wrong and the computer graphic simulated 
object will not appear to actually reside in the original photographed 
environment. 

When computer graphics generated objects or characters are placed in a 
scene and the lens do not match, the computer graphics generated objects fail to 
10 appear to be connected to their environment. The result may be large-scale 

motion artifacts such as vibrations or sliding, or the computer graphics element 
may exhibit inconsistencies compared to the original photographed image. For 
instance, characters that walk may appear to move an incorrect distance in 
A relation to their size or the size of their steps. Objects such as desks and cars 

m 

fx 15 which approximate known dimensions will not appear square and their 

perspective and angle will be incorrect; for example, the feet of a computer 
generated person may not stand on the floor. Thus, as a result of the object not 
correctly matching in the three dimensional space, visual disturbances occur to 
the viewer which can compromise the integrity of a computer graphic image. 
20 Modern photographic camera equipment and lenses contain a number of 

fixed and adjustable elements or parameters that need to be considered when 
providing a proper simulated camera system. The film gate (sometimes 
referred to as the aperture) represents a horizontal and vertical dimension of 
the image being exposed onto the photographic film or, in the case of the video 
25 camera, the size of the video image recording chip. The f-stop (sometimes also 
referred to as the aperture) on the lens controls the amount of light striking the 
aperture. The foca! length of the lens identifies the distance from the rear nodal 
point of the lens to the surface of the focal plane. The focus represents a 
distance in front of the camera. The field of view is the area photographed by 
30 the lens and contains the images captured through the lens. The circle of 
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confusion provides a measure of image clarity or sharpness of focus for a point. 
A camera typically has a focus ring to control focus from a setting of infinity to 
distances typically in the range of two to three feet. If it is a zoom lens a second 
control exists to manipulate the focal length. 
5 These different features have different scales and units to express their 

measurements. For example, the focal length typically is expressed in 
millimeters, the film gate (aperture) typically is expressed in thousandths of an 
inch, the actual film stock in the aperture typically is referred to in millimeters, 
the f-stop is a logarithmic scale based on light transmission through the lens, 
10 the focus is typically set in feet, or sometimes in inches or meters, and the field 
of view is typically expressed as an angle of degrees either horizontal, vertical 
i»2 or diagonal. In addition, the relationship between the horizontal and vertical 

!::~ dimensions of the aperture, referred to as the aspect ratio, is represented as a 

single number assumed to be a ration to a value of 1. Motion picture examples 
03 15 include 1.33:1, 1.66:1, 1.33:1, 1.66:1, 1.85:1, 2.20:1, 2.35:1. These typically are 
jj referred to as "1.33", "1.66", "1.85", "2.20", "2.35"; which represents how wide the 

s format appears to be to the viewer. In addition, the circle of confusion is 

!=ij typically measured in thousandths of an inch. 

fr\ A typical 3D software package does not take into consideration the effect 

20 of the lens on a computer generated image. Some 3D software packages, such 
as those marketed by SOFTIMAGE, an Avid Corporation, Inc. company, 
references some of the lens characteristics such as f-stop focus and the circle of 
confusion. However, the lens characteristics used are used strictly in relation to 
the creation of the combined computer graphic image as the final step (i.e., 
25 subsequent to the generation and combination of real world and computer 
generated images) in the simulation process. 

When objects are defined within a computer graphics software 
environment, a camera object is usually specified solely in order to provide a 
particular view of those objects. Since the system is an electronic simulation 
30 and does not use light rays or an optical system to capture and record the 
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image, physical real world issues like focus do not come into play. Instead the 
images are viewed by the user on the computer system or recorded out to a 
picture file. Thus, this computer graphics image is typically sharp to view. 

For these reasons, computer graphics images are normally sharp during 
5 the interactive phase when the images are generated and combined with other 
images, as the images presented to the user during that phase do not take lens 
characteristics into account. Typically an attempt to simulate the effects of 
focus and other lens artifacts are applied as part of a separate and last step, i.e., 
subsequent to the interactive phase, in the creation of a final computer graphic 
10 image. 

In addition, this methodology to simulate lens characteristics tends to be 
broad and rather imprecise by nature as it is based on a scaled down set of 
criteria and guidelines for lens and lens effects within the program. For 
instance, even though some type of depth of field simulation might be 

15 calculated during the final recording of a computer graphics image, the change 
in focus does not actually change the focal length such as happens with a real 
lens. In addition, any settings or effects are typically performed by user 
guesswork or trial and error. Thus, artifacts typically occur. 

In addition, it should be realized that such systems do not provide 

20 adequate feedback of values, determinations of exact boundaries or locations of 
lens effects. 

Furthermore, the calculations typically used to derive the final field of 
view for a particular lens in a 3D package contains assumptions, omissions, 
oversights and oversimplifications of the real world equivalent true lens and 

25 camera combinations. Most notable are the lack of solving for the effects of 
change in focus as it relates to focal length and the lack of equivalent controls 
compared to a real world camera. The relationship among lens attributes, such 
as focal length, focus and f-stop are not well understood or implemented in 
current software packages and do not successfully address the problems and 

30 details of simulating the changes in these optical characteristics. To appreciate 



this, consider that a 100 mm lens focused at 2 ft. turns into a focal length of 
nearly 120 mm. This is not dealt with by current programs. 

Thus, with respect to focal length, changes to focal length caused by the 
changes of other lens characteristics have been overlooked. Such effects, such as 
the changing of focal length as a lens is focused, are obscured to the observer by 
the pronounced effects of blurring as the object goes in and out of focus. In the 
situation where matching a real world environment to a computer-generated 
object is not exact, the situation is oftentimes only fixed through trial and error 
and a user may need to guess tolerances and measurements as an 
approximation to regenerate a combined image multiple times to see what 
"looks best". 

In addition, currently available 3D software packages do not contain 
some of the features common to many real world cameras, such as 
interchangeable reference clips and direct and interactive control of focus, zoom 
values and depth of field. These missing elements constitute important 
technical considerations for getting the exact lens setting as close as possible to 
the settings of the real world camera and for being able to operate a computer 
graphics virtual 3D camera in the same manner and ease as a real world 
camera. 

The digitally simulated camera and lens system and method of the 
present invention addresses the omissions, oversights and oversimplifications 
incurred in 3D simulation processes and addresses these problems by 
integrating into a computer graphic, virtual, three dimensional camera and lens 
model the variables and features of a real world camera and lens device. 



SUMMARY OF THE INVENTION 



The system and method of the present invention provides an innovative 
way to simulate true camera and lens devices. In one embodiment, camera and 
lens parameters input are used to generate true camera and lens parameters. In 
an alternate embodiment, if a camera and lens parameter is changed, 
corresponding camera and lens parameters are updated to provide a true 
camera and lens environment. For example, in one embodiment, if the f-stop is 
changed, the near and far focus values are is updated. In an alternate 
embodiment, the system and method of the present invention enables the 
combination of real world and computer generated images without undesirable 
visual artifacts that can occur in prior art systems. Since real world camera and 
lens can be accurately simulated, computer generated images can be generated 
having real world camera and lens characteristics, and a simulated camera and 
lens can be matched to a real world camera and lens device. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The objects, features and advantages of the present invention will be 
apparent from the following detailed description in which: 

Figure la is an overview of one embodiment of the system of the present 
invention. 

Figures lb and lc are simplified block diagrams of a desktop computer 
system and handheld device that operates in accordance with the teachings of 
the present invention. 

Figure 2 is a flow diagram illustrating the input, computations and 
output of one embodiment of the method of the present invention. 

Figures 3a and 3b are simplified flow diagrams illustrating the 
embodiments of processes for adjusting camera and lens parameters in 
response to a camera and lens parameter changes. 

Figures 4a and 4b are simplified flow diagrams illustrating embodiments 
of a process for generating a computer generated image in the field of view of a 
camera modeled according to camera and lens characteristics of a real world 
camera and lens. 

Figure 5 is a simplified flow diagram of a process for determining 
camera and lens characteristics in accordance with the teachings of the present 
invention. 

Figure 6a is a flow diagram illustrating the processes utilized in one 
embodiment of the system and method of the present invention and Figure 6b 
is a table illustrating parameters and variables utilized. 

Figure 7 is a flow diagram that illustrates the infinity cutoff decision 
process in one embodiment of the system and method of the present invention. 

Figure 8a is a perspective view illustrating one embodiment of the 
components of camera and lens system in accordance with the teachings of the 
present invention. 
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Figure 8b is a perspective view illustrating one embodiment of the 
components of the camera and lens system of the present invention. 

Figure 8c is a camera lens point of view showing the different camera 
and lens parameters in one embodiment. 

Figure 8d is a top view illustrating the camera and lens parameters in 
one embodiment. 

Figure 9a is one embodiment of a top view illustrating the field of view 
of a camera and lens system in accordance with the teachings of the present 
invention. 

Figure 9b is one embodiment of a perspective view of the camera and 
lens parameters illustrated by Figure 9a. 

Figure 9c is one embodiment of a camera view with superposed 3D 
charts and objects in accordance with the teachings of the present invention. 

Figure 9d is one embodiment of a camera view showing a 3D reference 
chart and objects without the superposed grid as illustrated in Figure 9c. 

Figure 10a shows a top view of an alternate embodiment of a display of 
camera and lens parameters in accordance with the teachings of the present 
invention. 

Figure 10b provides a perspective view of Figure 10a. 

Figure 10c shows a camera view of the camera and lens parameters in 
accordance with the teachings of the present invention. 

Figure 11a shows an alternate embodiment of a display of camera and 
lens parameters in perspective in accordance with the teachings of the present 
invention. 

Figure lib shows a top view illustrating the field of view of one 
embodiment of the present invention illustrated b Figure 11a. 

Figure 11c shows a camera view of a 3D reference chart and objects 
corresponding to Figure 11a. 

Figure lid shows the camera view of Figure 11a including 3D reference 
chart, objects and a superposed grid. 
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Figure 12a is a top view of an alternate embodiment of a display of 
camera and lens parameters in accordance with the teachings of the system of 
the present invention. 

Figure 12b is a top orthographic view of the parameters illustrated in 
5 Figure 12a. 

Figure 12c is a camera view of Figure 12a illustrating an object and 
reference chart. 

Figure 13a is a top orthographic view illustrating an alternate 
embodiment of a display of a camera and lens field of view in accordance with 
10 the teachings of the present invention. 

Figure 13b is a camera view illustrating 3D reference chart and objects of 
Figure 13a in accordance with the teachings of the present invention. 
;f=J Figure 14a is a top orthographic view showing the field of view in 

' J accordance with the teaching of the present invention. 

1 5 Figure 14b is a camera view showing a reference chart and objects in 

j ^ accordance with the teachings of the present invention. 

f.ij 

ry 
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DETAILED DESCRIPTION 

The digitally simulated camera and lens of the present invention, in one 
embodiment, provides numeric and visual feedback of camera and lens 
parameters and automatically updates camera and lens parameters when 
5 changes are made to one or more camera and lens parameters. Thus, the 

operation of the system is not only consistent with the operation of a real world 
or true camera and lens device, but exceeds real world camera operation with a 
precision not typically possible in the real world environment without a great 
amount of manual time consuming effort. The invention provides numeric and 
10 3D feedback of lens characteristics to help the user predict and visualize lens 
characteristics before photography takes place and recreate lens characteristics 
after images are created. It also provides a visualization and presentation of 
;P : lens data in a new and unique interactive fashion, including characteristics that 

J are difficult or impossible to see in real world or typical computer graphics 

;;o 15 conditions. 

1 75 Furthermore, the system of the present invention can determine camera 

and lens parameter values in an interactive fashion without requiring overly 

yj burdensome computations. Thus, the system is capable of providing data to 

the user in one form of a predictive mechanism to define the effects of a real 

I s ; 

20 camera and lens before the recording of a real or simulated image or a 

yj 

mechanism that recreates a specific setting from a particular camera and lens 
after an image has been captured through normal photographic or electronic 
camera and lens systems. 

Thus, computer generated images can be easily matched to real world 

25 images taken through a camera and lens device without the unsightly artifacts 
that occur when real world and computer generated images are combined 
using prior art systems. In the following description, for purposes of 
explanation, numerous details are set forth in order to provide a thorough 
understanding of the present invention. However, it will be apparent to one 

30 skilled in the art that theses specific details are not required in order to practice 
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the present invention. In other instances, well known electrical structures and 
circuits are shown in block diagram form in order not to obscure the present 
invention unnecessarily. 

An overview of one embodiment of the system of the present invention 
5 is illustrated in Figure la. A processing system 42 receives input either through 
user input 40 or external input 41, such as may come from a camera and lens 
device, and performs certain steps to provide numeric and /or visual feedback 
reflective of the lens and camera parameters of a corresponding digital- 
simulated system and lens that simulates a camera and lens as defined at least 
10 in part by the input. In one embodiment, the steps performed by the processing 
system are determined from instructions stored on media, such as a memory or 
^ storage device or received across a transmission media; these instructions are 

\P t executed by processing system 42. 

''■•■■■4 In the present embodiment, the digitally simulated camera software 

m 15 module 44 is interfaced with a three dimensional (3D) software module such as 
H one produced by SOFTIMAGE, an Avid Corporation, Inc. company. Specially 

= _ developed 3D software that generates 3D images on a display and the 

Lij corresponding numeric values also may be used. 

; S The processing system 42 outputs to an external device 45 and /or 

20 monitor display 46. In one embodiment, the monitor display 46 is embodied in 
a desktop or notebook computer system, an example of which is illustrated in 
Figure lb, and provides 3D images of different views as seen through the 
digital camera and lens system of the present invention. Alternatively, the 
display may be a simple non-graphical display that provides a numeric 
25 feedback of camera and lens parameters generated in accordance with the 
teachings of the present invention. In the latter embodiment, this may be 
embodied in a simple handheld device, such as a calculator device illustrated in 
Figure lc, which may be used in the field to enable a user to immediately 
determine camera and lens parameter values in view of input parameters and 
30 changes to parameters. 
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Figure 2 is a flow diagram illustrating further the operation of one 
embodiment of the system and method of the present invention. In particular, 
in the present embodiment, the input 50, which may be received from a user or 
from a physical device, may include such parameters as camera orientation 
5 data, lens attributes, film format and inserted reference objects. 

For example, camera orientation input data may include the XYZ 
translational position of the virtual camera in space, as well as the pan, tilt and 
roll attributes of the virtual camera at any particular spatial position. The term 
"pan" relates to the y axis rotation of the camera, producing a horizontal sweep 

10 effect of a scene, a "tilt" relates to an x axis rotation of the camera, producing a 
vertical sweep of a scene, and "roll" relates to a z axis rotation, producing a 
rotational or spinning effect of a scene. 

The camera film format input includes the aperture size and aspect ratio. 
The lens attributes may include the focal length, desired circle of confusion, 

15 focus setting, f-stop and infinity setting cutoff setting. In addition, reference 
objects can be input, such as clips, charts and visual aids, to assist the user in 
performing filming functions. As will be explained below, as parameters are 
input and /or modified, other parameter settings are updated correspondingly 
to provide a realistic and accurate camera and lens system. 

20 To provide this functionality, a number of calculations may be 

performed, such as shown in block 55, including true focal length, true field of 
view and hyper focal distance, depth of field, near focus limit and far focus 
limit, aspect ratio and film gate (aperture) setting. 

Hyper focal distance represents a special case of depth of field in which 

25 objects at infinity, as well as the nearest possible objects, are photographed with 
acceptable sharpness. Therefore, if a lens is focused at the hyper focal distance, 
all the image points between one-half that distance and infinity will not exceed 
a specific circle of confusion. The one-half distance is defined as the hyper focal 
focus. 
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Depth of field represents the range of distances from the camera within 
which objects will be considered acceptable sharp for photographic purposes. 
Depth of field can be specified as a "near focus plane" and a "far focus plane" 
with the actual focus point itself being somewhere in between these two areas. 
5 Anything between these two planes will be considered "in focus" as defined by 
the circle of confusion setting. 

Correct horizontal and vertical angles of the field of view are determined 
from the true focal length. The system also determines corrected placement of 
reference objects input and correct placement of camera and lens attribute 

10 markers for display (on a monitor or other device 60). 

Figure 3a is a simplified flow diagram illustrating one embodiment of 
the method of the present invention. At step 305, camera and lens parameters 
are input. In the present embodiment, the parameters that can be input include 
those set forth in Figure 2. The data input can be received from a device such as 

15 a real world or computer generated virtual camera and lens device, or a media 
device which stores or transmits input data values. A media device includes, 
but is not limited to, memory, data storage device, a computer system, external 
device, a user via a user input device (e.g., through keyboard or cursor control 
device and a graphical user interface), or a wired or wireless network including 

20 local area network and the Internet. At step 310, if a change of a lens parameter 
is detected, for example, if the focus is changed, then at step 315, other pertinent 
lens parameters are adjusted in response to the lens parameter change. As a 
specific example, in one embodiment, when the focal length is changed all the 
attributes that relate to the focal length are changed. These include the depth of 

25 field, the hyper focal distance, the hyper focal focus and the angles of field of 
view. In addition, objects used to indicate or locate these effects are also shifted 
accordingly. Alternately, if there is no change in input camera and lens 
parameters, or if an input parameter change does not offset other parameters, 
the process continues and is set to accept new input camera and lens parameter 

30 data. 
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Another embodiment is illustrated in Figure 3b. At step 320 camera and 
lens parameters are input. True camera and lens parameters are generated, step 
325, based upon the input. Markers or charts can be enabled /input to the 
system to be viewed by the user, step 330. 
5 These markers include markers defining near focus and far focus or 

other parameters such as the actual position of the hyper focal focus, position of 
the hyper focal distance, position of the focus itself and the position of the near 
and far limits of focus (depth of field). The system of the present invention may 
be configured to output these positions as distances or locations. Other types of 
10 markers and charts follow mechanical or physical versions in the prior art and 
are part of the comprehensive lens and camera simulation of the system of the 
invention. 

^3 Thus, if at step 335 a camera and lens parameter is changed, at step 340 

s.j pertinent camera and lens parameters are updated and at step 345 markers and 

ff, 15 charts are updated if necessary to provide a consistent view to the user such as 
would be viewed if a physical marker or chart is mechanically attached to a 
camera and lens device. 

In another embodiment, as illustrated in Figure 4a, a virtual camera can 
be modeled, enabling the simulation of a camera and lens system. At step 410, 
20 the characteristics of a real world camera and lens are input. This input may 
also not be reflective of an actual real world camera and lens device. At step 
415 a model of a digitally simulated model of the camera and lens is generated 
based upon the input camera and lens characteristics. At step 420 a computer 
generated image in the field of view of the modeled camera and lens or data 
25 representative of the same is generated. 

In this embodiment, the system can be used to compute camera and lens 
parameters in the field for correct adjustment of a camera and lens device for 
subsequent use in a computer generated camera and lens system, or to generate 
a true display of an image according to the camera and lens parameters input. 
30 In this situation, the system can be used to predict, visualize or define a 
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situation with the camera and lens before actual photography takes place. 
Thus, the system eliminates guesswork and a search for proper values when 
looking through a viewfinder while providing instantaneous updates as values 
are changed. 

5 In another embodiment, as illustrated in Figure 4b, the invention is used 

after a real world image has been photographed in order to better integrate the 
computer graphics camera with the real world camera, as well as duplicate 
properties of the real world camera and lens. At step 455, a real world image is 
input and at step 460 the lens and camera characteristics that photographed the 

10 real world image are input to the system. At step 465 a digitally simulated 
model of a camera and lens is generated based upon the input lens 
characteristics. At step 470 a computer generated image in the field of view of 
the modeled camera or data representative of the same is generated. In this 
embodiment, the system allows the computer graphics camera to more closely 

15 conform to the real world camera. 

The system of the present invention also enables the animation of 
parameters that are typically not animatable. On a typical camera and lens 
device, some parameters can be changed on a routine basis, for example, focus, 
while other things cannot be changed because of physical and mechanical 

20 limitations, such as the size of the film or the aperture. Because the system of 
the present invention provides a simulated environment that is not constrained 
by mechanical physical limitations, it allows for the animation of parameters 
that typically are considered unanimatable. 

At first glance, it would appear that such a feature would not provide 

25 any useful ability; however, during an image's life, the image may go through 
many processes, any one of which can cause the aperture or aspect ratio to 
change. The most typical situation would be if the film were scanned or 
digitized for being input to an image based software package. If the pictures 
are altered either through cropping or some other image processing technique, 

30 it would be possible to change the apparent aperture size or aspect ratio of the 
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digitally simulated camera and lens using the system of the present invention. 
If these values are changing throughout the course of a motion picture shot, the 
system would cause an animating change in the apparent aperture and /or 
aspect ratio of the images. Thus, the system of the present invention provides 
5 the ability to animate these attributes accordingly to compensate for a change in 
aspect ratio or him gate pass necessary to correct artifacts that may result from 
such an unusual situation. 

In addition, based on a portion of the lens parameters input, other lens 
parameters can be generated. This is illustrated in the flowchart of Figure 5. At 
10 step 505, at least a subset of camera and lens parameters desired are input. At 
step 510, the true lens parameters are generated based upon the input. 

In an alternate embodiment, the user or a device inputs a basic subset of 
parameters, which preferably are correct values, to initialize the process. Once 
the basic subset is input, the system is able to correct those lens and camera 
1 5 values based upon the relationship of the parameters to each other. Thus, for 
example, the system may correct a focal length for focus and adjust all 
necessary derived values, such as readjustment of the depth of field based on a 
new derived focal length determined from focus calculations. Therefore, based 
on the input, the system may adjust a parameter even though a value was 
20 input. As the camera and lens parameters of a real world device can be 

simulated by the process, the simulated camera and lens system can be matched 
to a physical camera and lens device. 

At steps 520, 525, corrected data is output. In one embodiment, the 
numeric values are output, step 525, to a display, storage device or other output 
25 device or media. Alternately, graphical representations, as will be illustrated 
below in subsequent drawings, are generated, step 520. In an alternate 
embodiment, both numeric values and graphical representations are output, 
ste£s 525, 520. 

JI^'w'^V more detailed flow diagram of the processes utilized in one 
30 embodiment of the present invention is illustrated in Figure 6a. Figure 6b is a 



i'i! 

i. ; 



\:'J 
li 

u 

fU 



17 




presenting a number of the variables that will be referenced in the 

discussion. Referring to Figure 6a, the input consists of focus 605, 
focal length 610, aspect ratio and aperture values 615, infinity setting 620, f-stop 
and circle orconfusion 625. Utilizing the focus 605, focal length 610, the true 
focal length 630vis determined. The true focal length may subsequently be 
reflected as a number provided to a user holding a field device, such as a 
handheld device, or graphically presented to the user as will be shown below. 
To determine the true foteal length, a computation or a look-up table may be 
used, block 635. In one embodiment, the calculation is as follows: 
TFL = FLV(Focus-FL), 

where the TFL represents the true fiscal length, FL represents the focal length 
>ut and focus is the Focus value ii 

sing mathematical equations may be sufficient for some simple lenses. 
Howeve^Very complex lenses, lenses that are of special design, or even lenses 
that are out o\alignment or unstable may require a measurement and mapping 
of basic lens properties from which look-up tables may be generated. This may 
occur, for example/when a particular type of lens and camera combination is 
required or some photography has been already shot with a particular type of 
lens in camera that was not in perfect condition. Furthermore, it is fairly 
common for the markings on a lens to be slightly off. For instance, the marks to 
indicate focus setting on a lens can be off by enough to cause an object to be out 
of focus if one were to only go on the lens markings and not look through the 
viewfinder. Even some lenses mature considered in "perfect" functioning 
condition simply do not operate according to the normal optical equations. 
There are a number of lenses, particularly zoom lenses and those designed for 
microscopic or macroscopic whose design is fundamentally non-linear. In these 
cases, changing lens attributes such as focus or the focal length (on a zoom lens) 
have unpredictable results. Thus, some mechanism needs to exist to adjust the 
recorded values so the values correspond to thevtrue optical properties of the 
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A look-up table provides a mechanism for incorporating those lens 
peculiarities into the invention's calculations to correct for these abnormalities. 

One advantage to the look-up table is that the table can provide for non- 
linear lens characteristics of a particular camera and lens device, thus enabling 
the system to better match a particular camera and lens device. The look-up 
table in one embodiment may be empirically determined. For example, a look- 
up table may be configured as follows to include empirically determined 
information: 



mark 


focal length 


focus 


i 
i 




inf 


2 


50.7630 


25 
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51.1501 


15 
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51.55 


12 
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51.9124 
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52.15 
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52.683 
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8 


52.8815 


5 


9 


53.7263 


4 


10 


54.16921 


3 


11 


55.87176 


2.5 


12 


57.136 


2 



A look-up table may be generated by typically sampling or measuring 
the changes in focus in focal length over a range of settings. Interpolation 
25 between these measured values is performed to produce correct lens values 
between the measured points. These values may then be used to provide a 
more accurate readout of the resulting changes. 

A correct vertical field of view 640 and horizontal field of view 645 are 
determined using the aspect ratio and aperture size 615. In this instance, the 
30 arctangent function is performed using a lookup table of values. 
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^h/^ TT^is, in one embodiment, the following computation is performed to 
*s determine th\correct horizontal and vertical field of view. 
Vfov = 2Wan(Ap/Ar)/2 / FL)) 
Hfov = 2*(ataK(Ap/2 / FL)) 
5 Where Vfov represents the vertical field of view, Hfov represents the horizontal 
field of view, atan represents an arctangent function, Ap represents the aperture 
size, Ar represents the aspect ratio and FL represents the focal length. 

Using the true focal length 630 determined, three dimensional clips or 
charts 650 can be placed a certain distance from the camera and subsequently 
10 can be consistently located regardless of other changes to lens parameters. This 
information is used to provide visual feedback 650 to the user in the form of 
lens parameter values or a graphical display of an image with the three 
dimensional clips and charts superimposed over the image relative to the 
•' j^articular field of view determined. 

one embodiment, the clips are positioned according to the following 

equation\ 

Clip Pos =\(TFL/2.54)/Ar 

W Where Clip Vos represents the reference clip position in feet from the camera 

rlj \ 

m position, TFLArep resents the true focal length, and Ar represents the film gate 

^ 20 aspect ratio. 

The hype? focal distance 660, is determined from the focus value 605 
input, the circle of confusion 625, and the true focal length 630 determined. In 
one embodiment, the hyper focal distance may be determined according to the 
following: 
25 Hd = (TFL 2 )/(Fs*Coc) 

where Hd represents the hyper focal distance, TFL represents the true focal 
length, Fs represents the f-stop and Coc represents the circle of confusion. 

Using the hyper focal distance 660, the hyper focal focus may be 
determined as: 
30 Hf = Hdx0.5. 
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ie focus 605 and true focal length are also used to determine three 
dimensional markers, such as scale and position 650, with respect to displayed 
image and lens characteristics. The hyper focal distance 660 also is used. In one 
embodiment, tne markers are placed in accordance with the following equation: 
5 Marker Pos = (TF^/2.54)/Ar 

The hyper focal distance 660 is used to determine the hyper focal focus 

670. 

The hyper focal focus 670 is used to determine the depth of field, in 
particular, far focus and near focus. In one embodiment, the far focus and near 
10 focus is determined as follows: 
Nf = (Hd*Fo)/(Hd+(Fo-TFL)) 
Ff = (Hd*Fo)/(Hd+(Fo+TFL)) 

where Nf represents the near focus, Ff represents the far focus, Fo represents 



SJ focus, Hd represents the hyper focal distance, TFL represents the true focal 

in 15 distance. 

! ;i A computer graphics environment is not typically bounded by the 

normal constraints of physical real world. Therefore, an object can be of nearly 
any size and any distance from the camera. However, this can bring out 
problems when dealing with objects or calculations that can often go to infinity 
20 such as the focus characteristics. 

For example, a number of situations arise where a particular focus 
setting and f-stop yield a far focus setting that is essentially set at infinity. This 
causes the far focus marker to shoot off to some astronomical distance from the 
camera in a simulated camera program; this is neither desirable nor does it 
25 provide any valuable information. In addition, some 3D software programs 
include a feature that automatically shifts the view or camera so all objects are 
in frame at one time. Thus, if an object is off at infinity, the top view in the 
three dimensional software will also scale back to an astronomical distance in 
order to try and "see" all the objects at once. 
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Unfortunately, if this happens, these view cameras move so far away 
that even large objects turn into small specs on the screen. The resultant effect 
is that valuable information is lost as the view moves back to include the far 
focus marker which is far away. Thus, in one embodiment of the system of the 
5 present invention, the user is able to "set infinity". The setting for infinity 
defines the far bounds of the area interest. The user can work at a reasonable 
scale with a user defined boundary or area while still gaining all the benefits of 
the mathematical calculations and feedback from the markers within that 
bounded area. 

10 This feature is valuable in a variety of situations, but it is particularly 

valuable when dealing with simulations, miniatures and models where the area 
of concern is very small. In this case, seeing all the markers requires careful 

^ setup of side, top and front views of the camera and focus marker positions and 

= y 

V» having a setting for infinity helps preserve these views. 

15 One embodiment of the process for determining the infinity cutoff is 

illustrated in Figure 7. The infinity setting 710 may be a single number set by 
the user in order to define the limits of the area of interest from the camera as a 
fjj base point. Therefore, if the user sets the infinity setting to 10 feet, then all 

::: : 

^ reference objects controlled by the invention will be restricted to movement 

M f 20 within 10 feet of the camera. No markers or other objects controlled by the 
invention will be allowed to go beyond 10 feet in any direction from the 
camera. If the user decides to change the infinity setting to 100 feet, everything 
is recalculated and all markers controlled by the invention will be restricted to a 
placement within 100 feet of the camera. This allows the user to restrict the 
25 invention's markers to a desirable range in order to limit their movement. 

Using the focus (Fo) 705, infinity setting 710, the hyper focal distance 715 
and true focal length 720 an initial far focus (Ff) is determined: 
Ff = (Hd*Fo)/(Hd-(Fo-TFL)) 

If the far focus exceeds the infinity setting 735, then the far focus is set to 
30 the infinity setting value 740. If the far focus does not exceed the infinity setting 
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and the far focus is less than the hyper focal focus 745, then the final calculated 
far focus value (FinFf)is set to be equal to the far focus value 750. Otherwise, 
the final focus value is set to the infinity setting 755. 

Embodiments and advantages of the present invention will be explained 

5 using the following figures. In one embodiment of the system of the present 
invention, it is contemplated that the values determined are generated as 
numerical values for a simplified presentation in a field use device, such as a 
small handheld device. In other embodiments, the graphical handheld display 
is generated; sample graphical display images are illustrated in the following 

10 figures. 

Referring to Figure 8a, a three dimensional perspective view of basic 
p camera and lens parameters is illustrated. The current view includes a camera 

;;!=! body 802, and lens 804. Immediately in front of the lens is a small rectangle 806; 

'M an arrow 808 points to the center of the rectangle 806. This is a 3D reference 

m 15 marker which is viewed by the camera 802. In this embodiment, the 3D 
J : /j reference marker includes the numbers "2.35" which indicate that it is a 2.35 

aspect ratio marker. The larger rectangle 810 is the near focus marker which 

t:j identifies one end of an acceptable focus towards the camera. In the one 

rl l 

'?p. embodiment, the letters NF (near focus) are associated with the rectangle both 

^ 20 horizontally and vertically for easy reference. Beside the letters and on top of 
the near focus marker rectangle, there is an arrow 811 pointing to the actual 
focus point setting 812. The tip of the arrow is exactly at this point of focus. 
Also included is a far focus marker (FF)814. This is similar to the near focus 
marker that defines the other end of acceptable focus towards the camera, and 
25 also includes an arrow 816 which points to the actual focus point 812. It should 
be noted that although the two arrows 811 and 816 are slightly offset from each 
other, they both point exactly to the same place in space, the actual user set 
point of focus 812. The values for the near focus marker and far focus marker 
are calculated from this point 812. Also illustrated is a hyper focal focus point 
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820. In addition, another arrow 825 points to the location of the hyper focal 
distance 824. 

In one embodiment of the system of the present invention, these markers 
may be instantly updated whenever an associated lens or camera parameter 
5 changes. For example, the markers will change with corresponding changes in 
f-stop or focus. Thus, the markers will shift towards the camera or away from 
the camera appropriately and can be viewed from a variety of angles, either 
through the camera or from another point of view. 

Figure 8b shows the same scene as Figure 8a, but the camera body has 
10 been removed and a reference grid 830 has been placed on the floor. For 

purposes of discussion, the grid illustrated in the Figures is included as a visual 
guide to better orient the viewer's perspective, to note the changes in the 
position of the markers as various lens parameters are changed, and to help 

='! : 

- "ST 

4 understand the 3D relationship of the camera, lens and marker objects as shown 

m 15 in the different views. 

[ J= ! In the embodiment illustrated by Figure 8b, the camera markers are 

attached to the camera and travel with the camera; therefore, no matter what 

ly direction the camera faces, or where it moves, the markers stay in correct 

orientation to the camera and lens. 

^ 20 Figure 8c shows the camera point of view for the scene of Figures 8a and 

8b. It should be noted that the aspect ratio or marker (2.35) is centered in the 
lens' field of view, whereas the depth of field is defined by the near focus and 
far focus markers. A top orthographic view can also be generated in the present 
embodiment. This is illustrated in Figure 8d. 
25 Figures 8a-8d show the same scene from three different view points. In 

this example, the user has set the focal length to 19mm and the focal distance is 
set to four feet with an f-stop of f2.8. Because focus affects focal length, the focal 
length, in accordance with the teachings of the present invention, has been 
recalculated to the true focal length of 19.3008mm. The values for the hyper 
30 focal distance, hyper focal focus, near focus, far focus and the 3D reference chart 



24 




E.i : 



. !=; 

-sis? 



position also are calculated to provide the correct relationship among the 
parameters. 

Figure 9a shows a wide scale top view with special reference lines to 
indicate the actual camera field of view 905, 910. The angle of view lines 915 

5 represents the exact edges of the camera's view. It should be noted that the 3D 
reference chart 920 exactly fits in the angle view at its distance from the lens. 
The chart does not extend beyond the angle of view lines, and does not fall 
short of the angle of view lines. The chart's three dimensional position from the 
camera is calculated to fit exactly in the field of view so that the reference charts 

10 always stay visually in the same place when viewed through the lens even 
though the lens values are changing. Thus, the chart is moved to compensate 
and maintain an exact steady position relative to the camera field of view such 
as a physical chart would do when physically attached to a camera and lens 
device. 

15 A one foot by one foot cube 925 is placed in the distance for comparison 

with other camera points of view. In Figure 9c the focal length is very short 
(e.g., a wide angle lens is used). Thus, the cube 925 appears very small. 
Figure 9b is a perspective view of the scene of Figure 9a, showing the 3D 
reference chart 920, the near focus 940 and far focus 945, defining the field of 

20 view, and the hyper focal distance 950. Figures 9c and 9d provide a camera 
view of the scene of Figure 9a. In particular, Figure 9c shows a 1.5 aspect ratio 
chart wherein the text marker is turned off and the Motion Picture Industry's 
standard 22 field chart (VIST A VISION) is substituted in the place of the 2.35 
chart. Figure 9d shows a 2.35 aspect ratio 3D reference chart 910. 

25 Thus, it can be seen that the charts are interchangeable and follow the 

same rules of placement and orientation regardless of which chart is chosen. 

Figures 10a through 10c illustrate a scene with the f-stop set to f5.6. 
Comparing Figures 10a -10c to 9a - 9c it can be seen that the focal length has not 
changed and the reference chart position has not changed because the focal 

30 length has not changed. However, lens parameters based on the f-stop have 
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changed. These include hyper focal distance, hyper focal focus, near focus and 
far focus. For example, by comparing Figures 10c and 9c it can be seen that the 
apparent size of the near focus marker in Figure 9c occupies a much smaller 
part of the camera field of view than it does in Figure 10c. This is because the 
5 near focus marker has been calculated to be much closer to the camera with a 
much larger f-stop in Figure 10c. Similarly, the far focus marker is further away 
from the camera in Figure 10c compared to Figure 9c. These changes and 
values demonstrate several simultaneous changes in focus parameters as only 
one value has changed, in this case the f-stop. 
10 Figures 11a through lid illustrate the change in the focus distance. In 

this situation, the parameters are set to the same as shown in Figures 9a 
f , n through 9d, but the focus has been changed from four feet to ten feet. Because 

the focus is changing, the focal length changes accordingly. In this situation, 

PJ 

4 focusing further away from the camera results in a focal length that is slightly 

m 15 shorter; thus, the calculated true focal length equals 19.1192mm, while at four 
f i feet it was 19.3008mm. Visually, the difference can be seen by comparing 

Figure 9b with Figure 11a. Figure 9b has a focus of four feet, and Figure 11a 

h j has a focus of ten feet. 

n i 

j;^ It should be noted that the hyper focal distance and hyper focal focus 

20 and the 3D reference marker position have only changed very slightly due to 
the very slight change in focal length from the affected focus. However, the 
positions of the near and far focus have changed dramatically due to the focus 
being set further away from the camera. 

In Figure 11a the perspective view of the camera and lens markers have 
25 been moved back a significant amount in order to show the far focus marker 
being so far away from the camera. Also, it should be noted that in Figure 11a, 
the arrows coming from the near focus marker and far focus marker still point 
to the same position space as the actual focus setting of ten feet. As noted 
earlier, these marker arrows expand in accordance with the positions of the tails 
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set at the marker positions and the tips of the arrow stay at the user focus 
setpoint. 

The changing angle of view can be seen by comparing Figures 9a and 
lib. The angle of view change is slight; however, by comparing the location of 
5 the angle of view with respect to the grid in the two figures a slight change of 
position can be seen. The change is due to a very slight change of focal length 
between a four foot setting and a ten foot setting on a 19mm lens. These types 
of changes, although very small, are what lead to problems in the prior art 
when attempting to align images when the focal length has not been correctly 

10 calculated to take into the account the effects of focus. 

Figures 12a through 12c show a different camera and lens setup in which 
a 100mm lens is set to a ten foot focus and f-stop of F2.8. Figure 12a is a top 
orthographic view. It can be seen that two arrows point exactly to the ten foot 
line on the grid. It should also be noticed that the 3D reference chart has been 

15 moved up to 2.7861 feet in front of the lens, whereas the position for the 19mm 
lens setup (e.g., Figures 9a through 9d) was a little over a .5 feet. This is 
because the angle of view has become very narrow and to fit the angle of view 
exactly, the 3D reference chart must be pushed out further beyond the lens to 
correctly be represented in the camera field of view. This effect also can be seen 

20 referencing Figures 12b and 12c. In Figure 12b, the 3D reference chart 1250 is 
far beyond the lens 1255. The near and far focus markers (not visible) are 
positioned approximately ten feet beyond the lens. Thus, referring to Figure 
12b it can be seen that the much longer focal length lens results in fixed cube 
1265 becoming much larger in frame (row positioned roughly between the left 

25 "2" and "4" markers on the numbered grid) compared to the cube in Figure lid. 
The cube in Figure lid is the very small square 1122 to the left of the center 
zero mark; this illustrates the effects of the changing focal length. 

Figures 13a and 13b show the effect of changing the focus to twenty feet 
while keeping the other parameters the same. The near focus and far focus 

30 markers are positioned with the tips of the markers located at exactly twenty 
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feet. The depth of view is barely one foot on either side. However, near focus 
and far focus markers appear very large in Figure 13b. They are actually very 
far away as shown in Figure 13a . 

Figures 14a and 14b show the effect of changing the f-stop from f2.8 to 
5 fl6. By increasing the f-stop to fl6, the near and far focus are expanded much 
further away from each other resulting in a much greater depth of view. The 
arrows of the markers still point to the twenty foot mark which is a user set 
focus setting. As the f-stop is changed by the user, the lens parameters 
automatically are updated and positioned correctly. It should be recognized 
10 that throughout the examples shown, whenever a reference chart, such as the 
numbered grid, is seen through the camera, it usually appears absolutely the 
same regardless of the focal length or other changes, as if there were a fixed 
object attached to the camera eye piece aperture, just like a physical chart that is 
mechanically attached to a real world camera. As discussed above, the effect is 
15 achieved by calculating dynamically the reference chart position in order to 

^3 reverse the effects of the changing focal length. Thus, the reference chart 

U 

> appears unchanging to the user and camera point of view while it is actually 

i : 

changing its position in the 3D world to simulate the static effect. Visually this 
effect is illustrated by comparing Figure 12c and 9c. Although the near focus 
20 and far focus and other markers are moving, the reference clip does not appear 
to change position in the camera point of view even though its position is 
drastically different. 

The invention has been described in conjunction with the preferred 
embodiment. It is evident that numerous alternatives, modifications, variations 
25 and uses will be apparent to those skilled in the art in light of the foregoing 
description 
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